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DETAILED ACTION 
Response to Amendment 

1 . In response to the Office Action mailed August 18, 2004, applicant submitted an 
amendment filed on January 18, 2005, in which the applicant traversed and requested 
reconsideration with respect to independent claims 1,4, 11, 15 and 22. 

Response to Arguments 

2. Applicant argues Shpiro does not disclose or suggest the use "model phoneme 
array information" as recited in amended claims 1, 4, 15 and 22. However, Shiprio 
teaches a reference audio specimen library that stores speech models, which are 
reference audio specimen for each of a multiplicity of phonemes, words and/or phrases. 

Applicant also argues regarding claim 11 that Shpiro do not disclose or suggest 
evaluating the degree of matching the learner's speech with the model using "a 
likelihood distribution plane", to evaluate how well the student's speech matches an 
optimum pronunciation. 

However, Shpiro teaches normalized audio waveforms of both the pre-record 
reference audio specimen and a student's attempted repetition in which a score 
quantifying the similarity over time between the repetition and reference audio specimen 
is displayed, to provide feedback to the student (figure 1 with column 5, lines 10-16. 
The distance between a student's repetition and a set of models are employed. That is 
the extent to which the student's repetition corresponds to the models (column 5, lines 
23-27). Also, the system, for example, compare the student's response specimen with 
a corresponding plurality of stored reference specimens, thereby to obtain a plurality of 
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similarity values, and may use the highest of these similarity values, indicating the most 
similarity, as the score for the student's response (column 7, lines 33-48). 

Therefore, Shpiro teaches a likelihood determination means for evaluating 
degree of matching of each said word speech information with a model speech on a 
likelihood distribution plane, which as applicant argued on page 12 of the remarks, that 
it evaluates how well the student speech matches an optimum pronunciation. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 

form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

4. Claims 1-22 are rejected under 35 U.S.C. 102(b) as being anticipated by Shpiro 
et al. (U.S. Patent No. 5,487,671), hereinafter referenced as Shpiro. 

Regarding claims 1, 4, 15 and 22, Shpiro discloses a foreign language learning 
device (figure 2, element 210), method, computer-readable medium and computer 
program (column 1 , lines 57-62), hereinafter referenced as a foreign language learning 
device, comprising: 

word separation means (figure 3, element 260) for receiving sentence speech 
information (phonetic unit), the sentence speech information corresponding to speech 
produced successively by a learner (student) when the learner utters a sentence 
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(studenfs utterance; column 7, lines 5-15) including a plurality of words (multiplicity of 
words; column 5, lines 36-40), to separate said sentence speech information (phrases; 
column 5, lines 33-41 ) into word speech information on the basis of each word included 
in said sentence (column 7, lines 5-15) using model phoneme array (column 5, lines 
33-41); 

likelihood determination means (figure 1, element 40 with figure 2, element 280) 
for evaluating degree of matching (similarity) of each said word speech information with 
a model speech (figure 1 with column 5, lines 10-16 and column 7, lines 43-48); and 

display output means (figure 1, element 30) for displaying, for each said word, a 
resultant evaluation (figure 1, element 40) determined by said likelihood determination 
means (figure 1 with column 5, lines 10-27). 

Regarding claims 2, 5, and 16, Shpiro discloses the foreign language learning 
device further comprising storage means (figure 2, element 120) for storing a model 
sentence to be pronounced by said learner (prerecorded speech models) and model 
phoneme array information which corresponds to said model sentence (multiplicity of 
phonemes) and concerns the whole of said model sentence (column 5, lines 33-41 with 
column 8, lines 2-7), wherein 

said display output means (figure 1, element 30) presents said model sentence 
to said learner in advance (figure 5A), and 

said word separation means (figure 3, element 270) includes 

phoneme recognition means (reference audio for phonemes) for recognizing said 
sentence speech information (words/phrases) on the basis of each phoneme 
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information (column 5, lines 33-41 with column 7, lines 5-15 and column 7, line 65 - 
column 8, line 7), and 

word speech recognition means for recognizing said word speech information 
(response specimen) for each said word according to said phoneme information 
(phonetic unit/phoneme) and said model phoneme array information after the 
separation (column 5, lines 33-41 with column 7, lines 5-15 and column 7, line 65 - 
column 8, line 7). 

Regarding claims 3, 6 and 17, Shpiro discloses the foreign language learning 
device wherein 

said phoneme recognition means (figure 1) includes phoneme likelihood 
determination means (figure 1, element 40 with figure 3, element 280) for determining 
likelihood of each phoneme information (most similarity) in said sentence speech 
information (student's response), with respect to each of phonemes that can be 
included in said foreign language (British/American dialect; column 7, line 34 - column 
8, line 14), and 

said likelihood determination means (figure 3, element 280) evaluates the degree 
of matching of each said word speech information (evaluating the student responses; 
column 7, lines 33-41) by comparing, on a likelihood distribution plane of phoneme 
information (figure 1 with column 5, lines 10-16 and lines 23-27 with column 7, lines 33- 
48) in said sentence speech information (figure 2, element 520 with column 9, lines 2- 
3), each word likelihood determined along a path followed when pronunciation follows 
a phoneme array exactly the same as said model phoneme array information (column 
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5, lines 33-41 and lines 57-65 with column 8, lines 6-14) with the sum of word 
likelihoods (figure 5B, element 530) determined along mistakenly utterable candidate 
paths from a speech waveform of pronunciation by the learner (graphic representations 
of the waveforms; figures 6-1 1 ). 

Regarding claims 7 and 18, Shpiro discloses the foreign language learning 
device further comprising the step of evaluating a resultant pronunciation by said 
learner after practice of the pronunciation (audio specimen to be practiced; column 8, 
lines 40-45), said evaluation (evaluating) made on the basis of each said phoneme and 
said word in said model sentence uttered (student's responses) by said learner 
(column 7, lines 34-48). 

Regarding claims 8, 12 and 19, Shpiro discloses the foreign language learning 
method wherein 

said step of evaluating a resultant pronunciation after practice thereof includes 

the step of displaying a vocal tract shape model (graphic representation of the 

waveform) for each said phoneme via a display unit to said learner (figure 1 , element 

y 

30 with figures 6-11). 

Regarding claims 9, 13 and 20, Shpiro discloses the foreign language learning 
device wherein 

said step of evaluating a resultant pronunciation after practice thereof includes 
the step of displaying, via a display unit (figure 1 , element 30) to said learner, a model 
voice print (figure 1, element 32) and a voice print concerning pronunciation by said 
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learner (figure 1, element 34). said voice prints being compared with each other to be 
displayed (figure 1 with column 5, lines 10-16 and column 9, lines 24-35). 

Regarding claims 10, 14 and 21, Shpiro discloses the foreign language learning 
method wherein 

said step of evaluating a resultant pronunciation after practice (figure 1, element 
40) thereof includes the step of displaying, via a display unit (figure 1 , element 30) to 
said learner, position of pronunciation by said learner on a formant plane (figure 1 with 
column 5, lines 61-65 and column 8, lines 2-7). 

Regarding claim 11, Shpiro discloses a foreign language learning device 
comprising: 

storage means (figure 2, element 120) for storing a model sentence to be 
pronounced by a learner (prerecorded speech models) and model phoneme array 
information corresponding to said model sentence (multiplicity of phonemes; column 5, 
lines 33-41 with column 8. lines 2-7); 

display output means (figure 1, element 30) for presenting said model sentence 
to said learner in advance (figure 5A); 

word separation means (figure 3, element 260) for receiving sentence speech 
information (phonetic unit) corresponding to a sentence pronounced by said learner 
(student; column 7, lines 5-15) to separate the sentence speech information (phrases; 
column 5, lines 33-41) into word speech information on the basis of each word included 
in said sentence (column 7, lines 5-15); 
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likelihood determination means (figure 1, element 40 with figure 2, element 280) 
for evaluating degree of matching (similarity) of each said word speech information with 
a mode! speech (figure 1 with column 5, lines 10-16 and column 7, lines 43-48) on a 
likelihood distribution plane (column 5, lines 10-16 and lines 23-27 with lines 33-48); 
and 

display output means (figure 1 , element 30) for displaying, for each phoneme and 
each said word, a resultant evaluation (figure 1, element 40) by said likelihood 

determination means (figure 1 with column 5, lines 10-27), 

said word separation means (figure 3, element 270) including 

phoneme recognition means (reference audio for phonemes) for recognizing said 

sentence speech information (word/ phrases) on the basis of each phoneme information 

(column 5, lines 33-41 with column 7, lines 5-15 and column 7, line 65 - column 8, line 

7), and 

word speech recognition means for recognizing said word speech information 
(response specimen) for each said word according to said phoneme information 
(phonetic unit/phoneme) and said model phoneme array information after the 
separation (column 7, lines 5-15 and column 7, line 65 - column 8, line 7), and 

said foreign language learning device further comprising pronunciation evaluation 
means for evaluating a resultant pronunciation after practice of the pronunciation 
(audio specimen to be practices; column 8, lines 40-45) for each said phoneme and for 
each said word in said model sentence uttered (student's responses) by said learner in 
a pronunciation practice period (column 7, lines 34-48). 
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Conclusion 

5. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

• Kojima (U.S Publication No. 2004/0215445) discloses a pronunciation evaluation 
system. 

• Shi (USPN 6,438,524) discloses a method and apparatus for a voice controlled 
foreign language translation device. 

• Minematsu (USPN 6,249,763) discloses a speech recognition apparatus and 
method. 

• Shannon (USPN 6,224,383) discloses a method and system for computer 
assisted natural language instruction with distracters. 

• D'hoore et al. (USPN 6,085,160) disclose language independent speech 
recognition. 

6, THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
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the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jakieda R Jackson whose telephone number is 
571 .272.7619. The examiner can normally be reached on Monday through Friday from 
7:30 a.m. to 5:00p.m. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Wayne Young can be reached on 571 .272.7582. The fax phone number for 
the organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 



JRJ 

June 9, 2005 




